home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Monster Media 1996 #15
/
Monster Media Number 15 (Monster Media)(July 1996).ISO
/
os2
/
lxlt113.zip
/
LXLITE.ENG
< prev
next >
Wrap
Text File
|
1996-06-05
|
27KB
|
516 lines
──────────────────────────────────────────────
LX lite - an compressor for OS/2 executables
──────────────────────────────────────────────
Dedication: To my little daughter Alice,
born 12 Feb, 1996 at 03:45.
1. Distribution
───────────────
This program is freeware. This mean that you can distribute it in any way,
except for commercial use. Commercial use is allowed ONLY with my personal
permission. To contact me see the the last section in this file. This also
means that you have NO guaranties that it will perform exactly what YOU
need. I cannot carry any responsability for any damages, lost of profit etc.
caused by use or inability to use this program.
However, you CAN use it to enhance any, even commercial, product. This is
not for YOUR benefit but rather for benefit of poor users who are BORED of
the big size of executables.
The program is written exclusively in Virtual Pascal, beta 3, with use of
its built-in 32-bit assembler. This is an excellent language which takes
full advantages of OS/2`s possibilities, is BorlandPascal compatible, and
have a powerful built-in optimizer. If you want the lxLite`s source code,
please mail me, but you have to tell me WHY do you need it, to avoid the
people which likes to re-sell other`s programs under their name.
2. Introduction
───────────────
I think all of us are really bored of the big size and reduced
functionality of all those modern executables written to run under OS/2
(windos too). I don`t understand why they are so big, because most
compilers, even IBM CSet generate a modest size code. For a widely known
example let`s take MultiMaint. What a complex task it performs that its
executable occupies more that 700K? I don`t understand. Moreover, WHY
duplicate (and triplicate) almost the same executable as it does (I mean
MultiSafe and IniMaint which comes along with MultiMaint). It performs a
nice work, but it is TOO big for the task it acomplishes. OS/2 kernel have
the same size. I cannot afford such a large pile of shit on my HD, so I
killed MultiMaint & C°. :`-(. Too bad for its author.
Here is a workaround for this. You can just pack the executable so it will
be twice smaller and still perform the same task. Alas, it will grab the
same amount of memory as original executable does - this is the fault of the
program`s author.
As far as I know there is only one program which does the same - REPACK
from IBM (EWS?). But compared to lxLite it gives larger files using the
same algorithm. For example, COURIER.FON from OS/2 v8.192 it compresses into
44K and lxLite into 34K. Feel the difference. BTW, LINK386+Resource
Compiler compiles COURIER.FON also into 44K-size file. This make me think
that they use the same common library.
I packed ALL my executables (incuding but not limiting to ?:\os2\*.exe,
?:\os2\dll\*.* and ?:\os2\dll\ibmnull;laserjet) and my system is stiil
working fine :-) One of lxLite users (Pavel Roskin) observed that lxLite
packs even os2krnl :-) This is a very nice feature to create a SINGLE OS/2
boot diskette.
3. Features
───────────
lxLite compresses the files in the same way as LINK386 does. There is NO
way of implementing a executable packing method other than those two which
Warp uses (or the only for 2.x). So, here is a brief description of those
two algorithmhs:
1. Run-length packing. This is generally the same method that Microsoft C
for DOS uses. It gives VERY bad results because the structure of executable
files aren`t suited for that kind of packing. For example, PCX files uses
approximatively the same packing method.
2. Kinda Lempel-Ziv algorithm. Lempel-Ziv is the same method which almost
all DOS executables packers use - LZEXE, PKLITE, PGMPAK etc. The method
standartized for OS/2 executable files is not the most effective (IMHO).
Other thing is that OS/2 executable have a different loading algorithm in
contrast to DOS executables - parts of OS/2 executables are loaded only when
they are needed. So, Lempel-Ziv dictionary cannot cross the bound of a
single page (4096 bytes). Because of this, it gives not such good results as
it can.
lxLite can use any of these two methods either for packing or unpacking.
Generally the second gives best compression rates, but may be (?) there are
some files on which first will work better. Because of this the default is
to try to pack page using both methods and then choose the smaller result.
lxLite can be used as well for unpacking the files that previously have been
packed - either by lxLite, LINK386 or REPACK from IBM.
What kind of files you can compress? The LX format is used almost
everywhere in OS/2: almost everything is in LX format. So, you don`t have to
limit only to .EXE files: you can pack .DLL, .PDR, .QPR, .DRV, .FON, .SYS
(Virtual Device Drivers (VDDs) in \OS2\MDOS) as well. You can run lxLite on
virtually any file: if it is not in Linear Executable (LX) format it will
refuse to process it.
You can also consider compressing your entire \OS2\*\ directory structure,
it will give you a lot of extra space and absolutely no overhead! The time
spent for decompressing executables is recovered by the time which system
does not spend for reading the executable from HD because it`s much
smaller! So, reboot from stand-alone OS/2 diskettes (you can boot from OS/2
installation diskettes (first two) then press F3), then change to drive
where your OS/2 is and type
\[path]\lxLite os2\*.exe os2\dll\*.* os2\dll\ibmnull\*.drv
and so on. Also you can compress now the executables which are run at
system startup, because when they are run you cannot modify their
executables and DLL`s.
lxLite version 1.00 and above can replace executable modules even if they
are currently in use. In this case it will warn you that module is already
in use by another process, and will propose you either to replace it by its
packed version or either to skip this module. Choose at your wish, but
keep in mind that the modules you have replaced are kept now (their old
versions) in memory, so they eat up your swapfile space. Better reboot as
soon as you have this opportunity.
Versions above 1.00 are supplied in two different executables: one named
lxLite.exe is the usual version for OS/2 v2.99, Warp and aboves. Other,
named lxLite2x.exe is meant for use in old versions of OS/2 (i.e. 2.x, NOT
1.x because 1.x doesn`t have the LX format). Delete it if you`re a OS/2 Warp
user.
Version 1.1.0 and above detects executables which contains some data after
the LX structure itself (i.e. what`s called in DOS overlay data). For
example Watcom`s binded executables (such as WCC.EXE versions >= 10.0) and
Watcom Visual Rexx executables have such structure. In this case lxLite
shows an warning message and asks for confirmation whenever you really want
to pack such executable. It is STRONGLY recommended that you backup the
executable in question before trying to do it because it is VERY possible
that it will become non-functional if something gets changed in it (because
lxLite does not change any of possible pointers in data binded to LX as in
VREXX executables).
4. Options
──────────
There are alot of options in lxLite. I simply like options :-) So, you can
configure almost anything in lxLite. Moreover, to protect the user from need
of writing the same options lxLite support multiple configurations which are
kept in a single file. lxLite comes with some example configurations
(`factory defaults`) which are listed below:
────────────────────────────────────────────────────────────────────────────
default: (loaded by default)
It is a `work` configuration. All parameters are set to maximize
compression. .BAK file creation is disabled. Note that this configuration
is ALWAYS loaded before any other options are in effect, so even /C{#}
option is executed AFTER default configuration is loaded.
fast: Lowest compression level, fastest executable loading. According
to IBM when data objects are aligned inside of file on sector boundary
(512 bytes) executable will load faster. I cannot observe a difference,
so try it at your own. If some data in executable was previously packed
it is not de-packed then re-packed.
fast2: Compress a bit better, but slower. Loads as fast as CFG#1 does.
ver2x: Optimal for pre-Warp versions of OS/2. Versions before Warp (3.0)
does not know of the Lempel-Ziv (/EXEPACK:2) method. DON`T PACK
EXECUTABLES WITH LEMPEL-ZIV`s ALGORITHM if there is a chance to run
your program on OS/2 v2.xx (except 2.99).
failsafe: Lempel-Ziv packing enabled. A `fail-safe` configuration.
All data are written just like LINK386 does, because of this creates a
bit bigger files than using default configuration. FOR WARP ONLY!
max: Tightest compression level. VERY SLOW! It is rarely needed to use
this configuration.
newStub: This is a particular configuration used to replace one DOS stub
in LX executable by another without altering anything else. You have to
specify a filename for new stub - this configuration only tells lxLite
not to depack old objects and not to pack unpacked objects.
minStub: This is a configuration which is linked to newStub and replaces
stubs in given files by minimal possible stub of `say-error-and-exit`
type. You cannot make it smaller (at least I doubt it), only if you
shorten the error message.
vdmStub: This configuration tells lxLite to replace in specified files
stub by a `run-from-VDM`-style one. This is also as short as possible :-)
Please read the /T{#} option description for further details.
info: Use this configuration to get information about executables (see
options /V+) without rewriting them.
────────────────────────────────────────────────────────────────────────────
To use a specific configuration use /C# where # is configuration`s name.
The settings are stored into file lxLite.CFG in the same directory where
lxLite.EXE resides. You shouldn`t care about paths - lxLite will always find
it. For example, to use `max` configuration run lxLite /cMax. For a detailed
description of .CFG file format see section right below the following.
Here is a detailed explanation of what each switch does. Note that any
switch which accepts '+' or '-' sign after it (to enable/disable the
action which switch symbolizes) can be used without anything after it - this
is accepted as '+'. So, /V+ is equivalent to /V.
■ /A{P|S|N{P|S}}
Set alignment method for first and rest of objects. First object can
be aligned on [P]age shift, [S]ector or [N]o boundary. Note that the
last option (No boundary) is never used by LINK386, but it works well,
and the LX format allows it. All objects except first MUST be aligned
at least on a PageShift boundary. PageShift is a value that is specified
in LX header. If you want to redefine it use /R# option.
■ /B{+|-}
Enable (+) or disable (-) renaming of original file into .BAK.
Note that this is a BETA version - that is why .bak files are
enabled by default.
■ /C{#}
Use configuration with ID = #
The two predefined configuration names are "default" and "unpack".
First is always loaded as lxLite starts and second is used when /X+
option is specified (NOT equivalent to /cUnpack).
■ /D{#}
Set exclu[D]e filemask. All files that passes given filemask(s) will
be bypassed by lxLite. All filemasks must be separated by ':' (because
it cannot be a part of filemask). For example /D*.zip:*.arj:*.rar switch
will instruct lxLite to avoid all .zip, .arj and .rar files. Default
lxlite configuration (/Cdefault) includes the [exclude] configuration
which instructs lxLite to avoid all executables which are known to
have problems when packed. Note that the /D switches are additive, so
/D*.zip /D*.arj /D*.rar is equivalent to the example above. To clear
exclusion list simply specify /D.
■ /F{+|-}
Forced repacking. Use /F+ to bypass `already processed` message, i.e.
when lxLite thinks that file was already processed and it really wasn`t.
This usually doesn`t happen, but can happen when you try to replace a
stub by another of the same size in a already packed file.
■ /G[X|D]{#}
e[X]tra / [D]ebug data will [G]o into specified file. When lxLite will
encounter an LX file which contains debug or extra data and you will
tell it to discard that data, before discarding lxLite will put it into
specified file. The {#} component can be as well an filemask, so you can
for example put extra data into files with the same name as original, but
with different extension and so on. For example /GD*.dbg will instruct
lxLite to write discarded debug information into file with the same name
as original but with extension ".dbg".
■ /I{+|-}
Force (+) lxLite to run at idle priority. This mean that lxLite will do
its work only when no other activity in system occurs (waiting for an
keyboard/mouse event etc). This is the best in my opinion choice because
you can run lxLite in background and it will not degrade almost at all
system performance. However if you`ll run an `badly-behaved` VDM
session which grabs all CPU time lxLite will completely stop. When run
with /I- option lxLite does not changes its priority (i.e. you can run
lxLite /I- via priority.exe program which starts programs at given
priority).
■ /L{#}
Instructs lxLite to maintain an log file. It contains only those
file which lxLite processes, the files that were skipped by some
reason will not be present there. If no file name is specified the
default will be lxLite.log in the home lxLite directory. Beside the
filename, the start and final file size is written into log along
with the problems (if any) that were encountered when processing
(for example: 'Executable has been used by another process and replaced')
■ /M{R{N|1|2|3}|L{N|1}}
Set packing method & parameters. Second character defines the method
to set-up: `R` stands for run-length (/EXEPACK:1) and 'L' for Lempel-Ziv
(/EXEPACK:2). The third character is the level of compression using that
method; if N is specified the method is disabled. Three levels of
packing are provided for run-length compression. The level 1 is the
fastest. It searches only for 1-character strings. For example, the
'AAAAAAAA' string will be detected and packed as 8, 1, 'A' while
'ABABABAB' string will be stored as unpacked text. Level 2 detects
strings of any length up to 16 characters long. The example above will
be encoded then as 4,2,'AB'. This is the default setting for most
'factory` configurations. And last, 3rd level searchs for ALL strings
of any length (up to page size/2 = 2048). This compress VERY slow and
seldom gives real results, so use it only when you really need it.
The Lempel-Ziv algorithm can be either disabled (/MLN) or enabled (/ML1).
When enabled it searchs for all matches using a relatively fast
hash-table, so there is no need in gradations by compression speed.
■ /P{+|-}
Enable (+) or disable (-) pause before each file.
The program shows the name of file which will be processed and
offers a choice to continue or to abort.
■ /Q
Query all configuration options. Basically it simply types a colored
version of lxLite.cfg file :-) Any other options (if any) are ignored.
■ /R{#}
[R]e-align pages on specific boundary. (#) must be a power of two.
This is an analog of LINK386`s /ALIGN: switch. Many of the programmers
don`t care that the default setting for /A: is to align each page
inside the executable on 512 (sector) bound. The result is a lot of
unused space inside the executable. You can `re-link` such files with
another /ALIGN: option. Default is /R:4. To force lxLite to behave
like LINK386 you must use /R:512 option. This is equivalent to /ASS :-)
option.
■ /S{+|-}
Show (+) or don`t show (-) configuration in effect.
This is useful for examining which settings are stored into .CFG file,
especially for linked configurations (see below). For example
lxLite /Cdefault /S will show the default settings.
■ /T{#}
Use specified file as new DOS stub. DOS stub is a little DOS .EXE file
linked to OS/2`s LX file which is usually used to type an error message
in the case if the executable is not run from DOS command line. Usually
this looks like:
This is an OS/2 executable module
Along with lxLite are enclosed two stubs: stub_min.bin and stub_vdm.bin.
First is the standard `type-error-and-exit` type, but it is slightly
smaller than usual stubs used by various linkers. The second is an stub
which starts a new OS/2 session and runs program from it again. If OS/2
is not detected it types the same error message and exits. The default
for stub_vdm.bin is to let OS/2 decide the type of your executable
itself. Alternatively, you can specify the type of session to be started
by stub_vdm.bin. For this you need any hex editor - find the pattern
`SesType->' in stub and replace the byte that comes after arrow (->) by
needed session type. OS/2 recognizes next session types:
00 - OS/2 session manager determines type (default)
01 - OS/2 full-screen session
02 - OS/2 windowed session
03 - PM application
04 - VDM full-screen session
07 - VDM windowed session
■ /U{+|-}
Enable (+) or disable (-) unpacking file before packing
lxLite know how to unpack any of two methods described, so default
option state is enabled. Disable it only if you wish to save time.
■ /V{+|-}
Verbose (show alot of file information)
This is a switch for curious ones :-) It displays a little information
about processed file (to get full information on .LX file use exeHdr.EXE
from OS/2 Programmer`s Toolkit).
■ /W{+|-}
Enable (+) or disable (-) the [W]riting of resulting file to disk.
By default it is enabled, but you can disable it if you wish to examine
the executable information (/V+) without having rewritten or repacked file.
Also this option can be useful for debugging your options.
■ /X{+|-}
e[X]pand (+) or pack (-) given files.
Use this switch to decompress files. lxLite can decompress files
which has been compressed by itself as well as by other programs which
uses standard methods (i.e. Resource Compiler, LINK386, REPACK). It
is NOT identical to /cUnpack option.
■ /Z{#}
This option will set the `threshold` for lxLite to help him determine
when stub is a `dummy` one and when it is a functional program. There
are a number of programs (for example, \os2\xdfloppy.exe) which runs
both under DOS and OS/2 - in such programs DOS executable is implemented
into OS/2`s LX as a DOS stub. By default (and using simply /Z) lxLite
considers all stubs bigger than 1024 bytes as functional ones, and
therefore on them the /T{#} option has no effect.
/?,/H
Show a brief help. This is useful when you forget a particular
switch from all that list :-)
────────────────────────────────────────────────────────────────────────────
The .CFG file format is as simple as possible: it is a plain ASCII text
file which you can edit using any editor which supports them (almost any).
Any characters after a semicolumn ';' are ignored, i.e. you can put comments
in configuration file like this:
; This is an comment
Any line begining with a word and column (ex: "word:") are treated as a
particular configuration. This word is used to identify the configuration.
This is an example of a configuration entry:
failsafe: /APP /B- /G+ /R4 /MR2 /ML1 /P- /U+ /V-
As you can see, after column you should write any switches just like you
do with lxLite itself. Recursive /C options are NOT ignored, so you can
link one configuration to another, but /C option is proceeded last, so from
two overlapping options will be in effect the last one. Just for example:
here: /app /b- /g+
there: /ass /b+ /p+
Running "lxLite /cThere" will be equivalent to "lxLite /app /b- /p+ /g+"
Note that lxLite will ignore any sequenced links to the same configuration
to avoid infinite-loop recursion. So, "lxLite /cOne /cOne" will load `One`
configuration only ONCE.
5. Error messages
─────────────────
Like most normal programs :-) lxLite can eventually generate error
messages. Some of them can appear in similar conditions, but caused by
different causes. Here is a short list of the most frequent errors:
* Invalid configuration file format
Self-explaining, I think :-)
* Error reading configuration file
This error is generated ONLY if the file is not physically readable.
The program does not generate an error if the configuration file is
missing.
* Error writing configuration file
Self explaining.
* error reading executable
This is generated if executable is physically unreadable.
* error writing executable
This is generated if executable cannot be written onto disk. The cause
can be the insufficience of disk space - lxLite does not check for this
particular error.
* invalid executable file format
The file is not in [L]inear [E]xecutable. Note that you can get this
message for files with .EXE extension in the cause they are in old,
'New Exe (bwhahaha)' format or DOS executable or winDOS executable.
* unsupported executable format revision
This error can happen (may be :-) if you try to process an executable
with other revision number than 0. OS/2 Warp works only with revision
0, so you will not normally encounter this problem.
* invalid word/dword ordering in executable
The executable uses little-endian byte order. Seems that it is not for
Intel platform machines.
* executable target is an unsupported CPU type
This happen if the target CPU is other than 286, 386, 486 or P5.
* executable target is an unsupported OS
This mean that the executable is not for OS/2. The windos 95 uses similar
format, but its magic number is not `LX` but rather `LE`, so usualy
program will abort with `invalid format` error code.
* unknown entry bundle type in executable
* unknown page flags in executable
* invalid object page detected in executable
It`s something about internal structure that lxLite doesnt know about.
Please mail me if you encounter such files.
* not enough memory to load executable
I doubt this error can happen in OS/2 :-) Rather a swap-file full fault
will occur. BTW, it`s a bad idea of IBM programmers to trap instead of
returning NIL pointer at memory request :-(
6. Todo list
────────────
Here is a list of features I plan to add in some future versions. Any
suggestions are welcome, to contact me see the last section.
* May be a feature to pack the LX stub in VX-REXX files. However, I doubt
it is really needed because the difference in stub size (before and after
packing) is about 18K - lxLite cannot pack the real program data because
it is 1) out of LX structure and 2) because it is encrypted in some way
and such data cannot be packed at all (even PKZIP fails to do it).
* May be a `extra-pack` option. I have an idea how to pack executables
as tough (and tougher :-) than DOS packers do, however such executables
will not work as usual - they will work always as `unlocked` executables
(see UNLOCK program) i.e. they will reside in swap file if not enough
memory will be available - this doesn`t mean that they will work slower
but this means that swap-file size will increase dramaticaly when such
executables will be launched). Tell me if you really wanna it - if I`ll
receive enough requests I`ll do it.
7. Known bugs and limitations
─────────────────────────────
Here are a list of executables which failed for some reasons to work
packed; avoid them.
* PMJPEG v1.5. However, even IBM`s REPACK cannot repack it, so I think
this is because of either a bug in a linker used to generate it (and its
executable has a strange structure) or a kind of debug protection
(executable checksum checking?)
* VX-REXX executables. The main body of such executables (REXX ciphered
code) is simply appended to a small LX stub. Such executables have an
unknown structure and after lxLite packs its stub they become completely
obsolete.
* Watcom C & C++ v>=10.0. Seems that Watcom likes to append garbage to LX
files. These executables are meant for running in both OS/2 and DOS sessions
and also have a strange structure.
8. Contacting me
────────────────
You can contact me via e-mail at:
FIDOnet: 2:5030/84.5 (prefered)
Internet: Andrew Zabolotny@p5.n84.r5030.z2.fidonet
bit@freya.etu.ru (seldom)
Enjoy,
_\ndy
I would thank Herwig Bauernfeind who contributed to German translation of
the documentation; he also mentioned some bugs which was (finally!) fixed
in this version. Alas he wrote the German version of documentation before
I added some changes to lxLite so it is (very little) out of date.
Here is a paragraph written completely by Herwig:
9. Translator
─────────────
As I really was very happy about Andys program, I asked whether he would
like a german translation of the documentation of version 1.01. Well, some
things sound a bit 'bumpy', but first of all, I am doing this as a hobby
only, secondly I am not professional programmer. I hope, it will help
anyway.
You can reach me by email (although this is quite useless, as I
contributed the translation only):
FIDOnet: Herwig Bauernfeind, 2:312/5.35
InterNet: H_BFD@fidonet.at (does not work reliably by now)